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Abstract 

An evaluation of an information retrieval system using a Boolean-based retrieval engine and 
inverted file architecture and WAIS, which uses a vector-based engine, was conducted. Four 
research questions in aeronautical engineering were used to retrieve sets of citations from the 
NASA Aerospace Database which was mounted on a WAIS server and available through 
Dialog File 108 which served as the Boolean-based system (BBS). High recall and high 
precision searches were done in the BBS and terse and verbose queries were used in the 
WAIS condition. Precision values for the WAIS searches were consistently above the 
precision values for high recall BBS searches and consistently below the precision values for 
high precision BBS searches. Terse WAIS queries gave somewhat better precision 
performance than verbose WAIS queries. In every case, a small number of relevant 
documents retrieved by one system were not retrieved by the other, indicating the incomplete 
nature of the results from either retrieval system. Relevant documents in the WAIS searches 
were found to be randomly distributed in the retrieved sets rather than distributed by ranks. 
Advantages and limitations of both types of systems are discussed. 
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1. Introduction 

Information retrieval has become an important issue in business and government due to the 
development of high-speed networking and finding aids such as Gopher, World-Wide-Web, 
Mosaic, and WAIS (see Obraczka, Danzig, & Li, 1993 for an overview of different systems). 
Such systems have provided broad ranges of computer users generally easy access to Internet 
resources, but access is not sufficient to assure information-seeking success. There is no 
evidence on how such systems perform either quantitatively or qualitatively and no guidance 
on what information seeking strategies are appropriate. Although the information retrieval 
(IR) community agrees that networked resources are important research and development 
domains, there have been no systematic studies of retrieval performance in networked 
environments. Practice rather than theory is driving development. This situation is problematic 
since networked end-user environments have enormous implications for IR researchers and 
information specialists. To explore how one such system performs, a comparative 
investigation was conducted for the WAIS system and a Boolean-based retrieval system. This 
report presents results from this study and makes suggestions for developers, evaluators, and 
users. 

1.1. Boolean-based Systems 

The development of systems to store and retrieve bibliographic records has been dominated 
for the past 30 years by the model that uses a Boolean-based retrieval engine and inverted file 
architecture to match query terms to document terms (these will be referred to as Boolean- 
based systems (BBS)). A search for relevant records is conducted in one of two modes: (1) 
using designated descriptors, typically including assigned index terms, authors names, and 
dates of publication; (2) using the free-text capability, which searches for individual words 
and phrases within citations. Among the prominent examples of such systems are Dialog, 

BRS, Medlars, and NASA RECON. 

These systems support the bulk of IR practice because there is a large installed base of 
databases and a community of professional intermediaries who have learned to maximize their 
value. Over the years these systems have added new features and tools so that professional 
intermediaries can have even better control in mapping user needs to system capabilities. 
Today’s BBSs allow skilled searchers to tightly control the scope and format of results (e.g., 
high recall or high precision; specific fields or full records), and new features such at 
Dialog’s Target (Dialog, 1993) offer new search capabilities based on frequency of word 
occurrence as well as traditional exact match retrieval. The specialized features make these 
systems powerful but require that users be highly trained to use their rich command 
languages. 

To attract end users, BBS vendors offer alternative systems that feature menu-driven 
interfaces, online thesauri, online help and abbreviated command sets (e.g.. Knowledge Index, 
BRS AfterDark, Grateful Med). Thus, BBSs continue to evolve from the rigid, exact-match 
systems often criticized in the research literature to diverse collections of interfaces that aim 
to serve professional and novice searchers. 
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Although these systems continue to evolve, BBSs are often criticized in four regards (e.g., 
Larson, 1992): 1) users must use Boolean logic to formulate queries; 2) Boolean operators are 
either too coarse (OR is too inclusive) or too fine (AND is too restrictive); 3) query terms are 
treated equally regardless of their relative importance to the user’s information need; and 4) 
all retrieved documents are treated as equally relevant (documents are not ranked). 

1 . 2 . Frequency of Occurrence-based Systems 

An alternative approach to information retrieval uses a vector space model for documents and 
queries (Salton & McGill, 1983). The basic idea behind the vector model is to represent every 
document as an N-dimensional vector where N represents the number of words in the index 
language (this can be a small set of controlled terms or, in the extreme case, every word in 
the document collection) and each cell contains a value related to frequency of occurrence of 
that cell's index word in the document. Thus, a document that does contain the word 
"thermal" but does not contain the word "radiation" would be represented by a vector that 
contains a 0 in the cell for the index word "radiation" and some non-zero value, dependent on 
the weighting scheme used, in the cell for the index word "thermal." Queries are likewise 
represented as N-dimensional vectors, and each document vector is compared to the query 
vector according to a similarity metric (e.g., the dot product or the cosine of angle between 
the vectors). 

Although computationally intensive, this type of representation allows all documents in the 
collection to be ranked in similarity to the query, thus mitigating the exact match limitations 
of Boolean-based approaches. Research related to this approach to information retrieval 
considers parameters such as types of term weighting schemes, types of similarity measures, 
size and composition of the indexing language (e.g., highly controlled or all words in the 
database), treatment of word phrases and proximity limits, and types of stemming. 

In addition to the SMART studies conducted by Salton and his colleagues, Harman and her 
colleagues have demonstrated the usefulness of the vector approach (Harman & Candelea, 
1990). Harman, McCoy, Toenese & Candelea (1991) argue that the frequency of a term's 
occurrence within the document taken in relation to its frequency of occurrence across the 
entire record file and in relation to the length of the document provides an accurate indication 
of the importance of the term in the documents and query. In this way, the system 
numerically determines that the term "AIDS" has great meaning for occurrences within a 
psychological counseling database, while being relatively unimportant within an AIDS 
database. Of course, this example also illustrates one of the dangers of both word matching 
and word counting approaches to full-text databases-homonyms can lead to irrelevant 
documents being retrieved in the BBS case and to misranked documents in the vector case. 

Based upon the term weighting values and the relationships that document terms share with 
users' query terms, vector-based systems ordinally rank retrieved record sets in order of 
descending similarity to the query. The first citation presented to the user is the one with the 
highest similarity score. In theory, the system retrieves the entire set of documents that have 
any of the query terms (equivalent to a OR of all free-text terms in a BBS) ranked according 
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to their similarity to the query. In practice, cutoff values are used to return only the 
top-ranked documents. 

Another information retrieval technique is known as relevance feedback (Ide, 1971; Rocchio, 
1971). This approach displays document citations to users and invites them to inform the 
system which citations are relevant. The relevant citations are then used to locate similar 
documents. Each iteration of the search is an opportunity to further refine the search by 
allowing the system to readjust the query vector by including and/or discarding selected 
terms. The user can proceed with as many iterations as necessary to retrieve those documents 
which best meet the perceived information need. Although either the Baleen or vector 
techniques may be used with relevance feedback, it is most often associated with the vector 
approach since term weights can be adjusted in the query 1 . 

The present study evaluated and compared the performance of a traditional system used for 
aeronautical engineering databases and a system based on vector retrieval and relevance 
feedback. Since these two systems take advantage of such divergent technical processes, 
representations of user queries, and methods for displaying documents, the selection of 
evaluation measures and the attributes used for comparison were carefully chosen and applied 
to capture the strengths and weakness of both types of systems. 

1.3. NASA/RECON 

NASA's implementation of RECON is one of the oldest of the large online bibliographic 
retrieval systems. In operation since 1 969, it holds the special status of being the 
government’s first large online retrieval system. A BBS, the system offers a wide range of 
commands for formulating queries and specifying the format of output. The system provides 
access to a body of citations to the scientific and technical literature for its users, who are 
primarily aerospace researchers and information specialists within the U.S. Government. In 
1993, the database held about 3,000,000 citations organized within 29 files, which are 
grouped into 17 clusters. The largest of these are Scientific and Technical Aerospace Reports 
(STAR) and International Aerospace Abstracts (IAA). 

1.4. Wide Area Information Server (WAIS) 

WAIS is the name of a suite of programs using the client/server architecture and based on the 
NISO Z39.50 information retrieval protocol (Kahle & Medlar, 1991). This protocol allows 
searchers using a variety of interface “clients” to formulate queries which are translated into 
appropriate formats for particular “servers” that contain the bibliographic or primary 
information. A WAIS client allows users to formulate queries in “natural language” (terms or 
phrases can be typed in or cut and pasted from other sources in some clients) and to select as 


1 This automatic refinement technique is similar to the traditional manual citation pearl growing technique used 
in online searching wherein the user studies the characteristics of those valued retrieved documents ("pearls") and 
then attempts to retrieve more citations like them by adding the appropriate index terms to the query statement. 
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many databases (servers) as they wish for concurrent searching. A WAIS server is a retrieval 
engine that uses statistical ranking algorithms to return lists of information objects (typically 
documents, but possibly images, program code, etc.) that are ranked according to weighted 
frequency of occurrence. WAIS clients also invite users to provide relevance feedback to the 
servers by selecting relevant documents, phrases, or sections from the retrieved objects. Terms 
from these objects are fed back to the server and new ranked lists are sent to the client. A 
variety of public domain clients are available for different platforms (e.g., MS-DOS, 
Macintosh, X-windows), and there are public domain as well as commercial servers. Thus, 
WAIS searching requires high-speed connectivity to be effective and allows users to search a 
variety of servers without specialized training in query languages. Many WAIS servers are 
available through the Internet, and the client and server software packages are undergoing 
continued modification. 

The proponents of WAIS system architecture make the following arguments: 1) The 
information search is driven by user perceptions of relevancy instead of the system's 
definition translated as an exact match between index and query terms. This distinction has 
been cast as the tension between relevancy, a match of document to query term, and 
pertinence, a match of document to actual information need. 2) The query representation 
automatically incorporates changes in the user's relevance judgments into the query vector and 
the order of the retrieved sets as the search proceeds. 3) The search process is made relatively 
transparent, allowing the user to concentrate fully on the intellectual demands of the 
information task instead of manual query refinement required by traditional systems. 4) 
Browsing by the user in the retrieved document sets can improve the final set's quality in 
terms of precision and recall and make it more reflective of the user’s changing perception of 
the information need. Bates argues that this technique of systematic iterative retrieval, or 
“berrypicking," reflects how users actually approach the retrieval task and increases user 
understanding of their information need (Bates, 1989). 5) Searching different databases 
simultaneously insures a radical expansion of the retrieval engine's power and, probably, of 
the subsequent end value of the delivered document set (e.g., Stanfill & Kahle, 1986). 

1.5. Study Perspective and Research Questions 

As these brief descriptions imply, there are profound differences between BBS and WAIS in 
both surface and deep structure. Keen (1992) said, "Comparisons between Boolean and 
non-Boolean are not commonly attempted, and encounter difficulties with finding a control 
variable fair to both systems." Additionally, there are no accepted metrics for assessing the 
effectiveness of interactivity, an especially key aspect of WAIS. The approach that was taken 
for this work was to explore the differences between the two systems by controlling the 
database and search questions to ascertain strengths and differences for both approaches. 

There is no overarching agreement within the information science field on the most 
appropriate measures to invoke and methods to use in evaluating an information retrieval 
system. Two measures that have been central to the debate about evaluation are recall and 
precision . Recall is the ability of the system to present all relevant items and is defined as the 
ratio of relevant documents retrieved to the number of relevant documents in the database. 
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Precision is the ability of the system to present only relevant items and is defined as the ratio 
of relevant documents retrieved to the total number of documents retrieved. It is generally 
recognized that recall and precision are inversely related and, thus, comparisons must be made 
at different levels of each. 

The typical criticism of recall and precision as performance metrics relates to the problems of 
relevance (McCam & Lewis, 1991). For example, 1) How does one determine the relevant 
items from those contained within a large information storage and retrieval system? 2) How 
can one be sure that users really care about relevant items not retrieved? 3) By what criteria 
should relevance be determined and to what degree are these judgements reliable? Beyond the 
individual interpretations of what is relevant to a question, even order effects have been found 
to influence judgments (Eisenberg & Barry, 1988). Nonetheless, recall and precision have 
become standard metrics for IR studies and were the first point of comparison for the work 
summarized here. Recall is the most troublesome value to compute in practice since it 
requires knowledge of the number of relevant documents in the entire database. Since this is 
impractical in working environments, this value is derived from extrapolation or estimates. 

For a system like WAIS that ranks documents and has quite severe cutoff levels (50 
documents in the client used for these studies), recall is particularly problematic since not 
only must extrapolations be used to determine the number of relevant documents in the 
database, but adjustments for the cutoff limits must be invented. Additionally, recall has been 
questioned as a usable metric for large databases since there seems to be little overlap in 
relevant citations retrieved by different queries (Belkin, Cool, Croft, & Callan, 1993). Given 
these problems, recall was not used as a dependent measure in this study. The raw number of 
relevant documents is used for gross comparisons, and precision values are used for 
systematic comparisons. 

Other criteria have been offered for IR evaluation. For example, Cleverdon (1964) not only 
used recall and precision but discussed effort (defined as the effort required by users to 
conduct the search), time (defined as the time necessary to generate the complete set of 
retrieved items), presentation, and coverage. Other possible criteria include cost, system 
leamability, system usability, and searcher satisfaction. Since the focus of this study was on 
retrieval performance, these criteria were not systematically assessed. 

Two main questions guided this study and spawned more specific questions as data collection 
took place. These questions were: 

• How do WAIS and the BBS perform when high recall, or the retrieval of all 
relevant documents, is the goal of a search? 

• How do WAIS and the BBS perform when precision, or the retrieval of only 
relevant documents, is the goal of a search? 

The end goal of the study was not to "prove" which is the "better" system. Rather, it was to 
provide a context wherein each system could display the features and capabilities which 
distinguish it from its counterpart. To this end, comparisons were used as the basis for 
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discussions of the two different systems and as a springboard for gaining a better 
understanding of the particular values and limitations of both Boolean- and vector-based 
methods of information retrieval. 

2. Methodology 

2.1 Database and Search Systems 

The database which served as the raw material for the two retrieval engines consisted of 
records in the Aerospace Database dated within the three year period 1990-1992. The 
Aerospace Database contains records for publicly available documents from NASA's STI 
Database. This set of documents consisted of approximately 205,000 document records which 
contained about 32 million words, about 540,000 of which were unique. 

This database was mounted on a WAIS server (commercial version 1.0.10) on a Sun 
workstation at NASA's STI Program Office. The database used for the work reported here 
was not stemmed, so all results are based on exact matches of words in queries, a possible 
disadvantage from a recall perspective 2 . The WAIS database was accessed over the Internet 
via an ethemet connection using a public domain Macintosh WAIS client (version 1.1). This 
client offered a maximum cutoff value of 50 documents, but consistently returned a maximum 
of 45 documents. 

As an example of a BBS, the Aerospace Database was accessed via the DIALOG network 
(Dialog File 108) using a MS-DOS platform and 2400 baud modem. All searches were 
date-limited to the same three year period as the records contained in the WAIS server. 

2.2. Search Questions 

Volunteers from among graduate students in aeronautical engineering were solicited to 
participate in the study. Their role was to supply questions (one question from each 
participant) for search and to decide upon the relevance of retrieved citations. Four students 
volunteered. Each was at a different stage in his research project; their questions varied in 
subject, complexity, and specificity. Two reference interviews were held with each participant. 
Prior to the first interview, the students sent an initial statement of their questions to the 
research team. These questions are summarized in Figure 1. During the first interview, they 
were asked to state their questions again and to note any synonyms or related terms. They 
were also asked about the usefulness of sources already consulted. 

2.3. Procedure 

The first step in the search process was to retrieve as complete a set of relevant documents as 
possible from the database. This was done by conducting exhaustive high recall searches on 
the Aerospace Database accessed through the Dialog system. Search terms were taken from 
the participants' queries, the print version of the NASA Thesaurus (1988 edition with 


2 However, Hannan (1991) has demonstrated that stemming offers few advantages when ranking 
algorithms are used. 
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updates), and the online thesaurus in Dialog, which supplied the newest indexing terms. Free 
text searching was used as needed. For each of the four questions, a search strategy that 
would maximize recall was developed. Experienced online searchers from outside the project 
reviewed the strategies and offered suggestions for improvement. 

Two searches in the Aerospace Database accessed through Dialog were conducted for each 
question. The first search was exploratory, incorporating terms in the original query 
statements and from the NASA Thesaurus. The second search expanded the terms used to 
include as many applicable terms as possible. In one case (question 3), a set of about 2500 
citations was limited by NOTing out the common phrase “computational fluid dynamics” so 
that the set provided to the requestor was more manageable. These searches are referred to as 
the BBS recall searches. See Appendix A for search strategies. 


Figure 1 . Original Search Questions 

Subject 1 . The forces of pressure used to predict what the effect would be on a wing using 
computer models. 

Subject 2. Basic Objective: Design of Flight Control System for Flexible Aircraft using 
Modem Control Theory. 

Some Necessary Keywords (or Key Sentences) are Application of H-Infinity & H-2 
Control Theory to Flight Control System; Development of Flexible Aircraft; Dynamic 
Model for Flight Control Design; Modem Control Theory & Its Implication to Aircraft 
Flying/Handling Qualities; Application of Active Control Technology to Transport Aircraft; 
Stability and Control of Control Configured Vehicle. 

Subject 3. Adaptive gridding on structured grids. Other keys words I am interested in are 
redistribution and truncation error." Some key words which are not of interest are 
"unstructured grids” and "refinement." 

Subject 4. Computational Fluid Dynamics. I am trying to model flow over a bluff body, 
(something like the automobile side view mirror). I am using a computer code which 
solves the equations governing fluid dynamics. This branch of study to simulate flow using 
computers is called Computational Fluid Dynamics as opposed to conventional wind tunnel 
testing. I usually look for following Key Words: Unsteady, Incompressible, Bluff Body 
flows; High Reynolds number Flows, Turbulence Modelling; Finite Analytic Method; Non- 
Stationary turbulence; Ground effect; Unsteady pressure in incompressible flow. 


A member of the research team met with each of the subjects to present the retrieved citation 
set. The team member asked the participant to rank the relevance of each citation, using a 
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five-point scale with 1 as "Not Relevant" and 5 as "Highly Relevant." The citations rated 4 
"Relevant" and 5 "Highly Relevant" formed the first component of the set of relevant 
documents in the database for the three-year period. 

Subsequently, BBS searches were conducted to maximize precision rather than recall. These 
searches were constructed and modified to focus on documents containing all facets and are 
termed the BBS precision searches. See Appendix B for high precision search strategies. 

The same four questions formed the basis of searches in WAIS. Initially, three searches were 
run on each question. The first search was a keyword search using an abbreviated set of terms 
from the query; in the discussion that follows this search is called the Terse search. These 
queries ranged in length from five to eleven terms. The second search was a keyword search 
using a more inclusive set of terms; it is called the Verbose search. These queries ranged in 
length from 25 to 42 terms. See Appendix C for the WAIS search strategies. The third search 
was a Boolean search, formulated as close as possible to the Boolean searches performed in 
Dialog, although it was impossible to duplicate the searches exactly. Only one Boolean search 
was successful; the other three searches did not produce usable results (timed out) in spite of 
multiple efforts. Because there was data from only one Boolean search, the data were not 
included in the analysis. 

The citation sets retrieved using WAIS were combined into a master set for each question. 

The participants were asked to rate the relevance of each citation in the set, using the same 
five-point scale. In each of the four cases, the WAIS searches produced citations rated as 4 
"Relevant" or 5 "Highly Relevant" that were not retrieved by the BBS searches. Seventeen 
items retrieved by both systems and for which the relevance judgements differed enough to 
affect the interpretation were eliminated from the database. The resulting sets of citations are 
the basis for the data analysis. 

Two additional WAIS searches were run on each question. These searches utilized the 
relevance feedback function of WAIS in order to refine the retrieved set of citations. In each 
case, up to five citations from the WAIS search that were rated "Very Relevant" were 
designated as example citations, and the search was run again. Since there are many 
variations on how relevance feedback may be used and it is most appropriate for end users 
themselves to make judgments about what documents to feed back, this procedure must be 
taken as exploratory at best. 

3. Results 

3.1. Comparison BBS and WAIS Results 

3.1.1. Comparisons Based on Precision 

The distribution of citations by relevance judgement for all searches is shown in Table 1. The 
data for relevance judgments were recoded to produce three categories, "Not Relevant" (1 or 
2), "Somewhat Relevant" (3), and "Highly Relevant" (4 or 5). For each question, the number 
of citations and number judgments in each relevance category are given for the BBS recall 
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search, BBS precision search, WAIS terse search, and WAIS verbose search. Note that 
percentage values in the “high” category represent precision values. 


In all questions, the precision results for the WAIS searches fell between the high recall and 
high precision results for the BBS searches. Since in practice recall and precision are 
inversely related, it would be highly surprising if a system that ranks output and uses a cutoff 
limit did not offer better precision than a high recall BBS search. 


Table 1: Relevance Judgments by Search 


QUESTION 1 

NOT 

SOME 

HIGH 

TOTAL 

RECORDS 

BBS Recall 

193 

(80%) 

18 

(7%) 

32 

(13%) 

245 

BBS Precision 

2 

(25%) 

0 


6 

(75%) 

8 

WAIS Terse 

15 

(34%) 

7 

(16%) 

22 

(50%) 

44 

WAIS Verbose 

20 

(44%) 

8 

(18%) 

17 

(38%) 

45 

QUESTION 2 

BBS Recall 

57 

(12%) 

257 

(52%) 

176 

(36%) 

490 

BBS Precision 

3 

(18%) 

0 


14 

(82%) 

17 

WAIS Terse 

4 

(10%) 

21 

(53%) 

15 

(38%) 

40 

WAIS Verbose 

4 

(10%) 

17 

(44%) 

18 

(46%) 

39 

QUESTION 3 

BBS Recall 

735 

(93%) 

26 

(3%) 

43 

(5%) 

804 

BBS Precision 

9 

(39%) 

3 

(13%) 

11 

(48%) 

23 

WAIS Terse 

21 

(48%) 

12 

(27%) 

11 

(25%) 

44 

WAIS Verbose 

36 

(80%) 

4 

( 9%) 

5 

(11%) 

45 

QUESTION 4 

BBS Recall 

196 

(81%) 

10 

(4%) 

35 

(15%) 

241 

BBS Precision 

26 

(68%) 

0 


12 

(32%) 

38 

WAIS Terse 

30 

(67%) 

5 

(11%) 

10 

(22%) 

45 

WAIS Verbose 

18 

(40%) 

23 

(51%) 

4 

( 9%) 

41 
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As expected, the precision values for the WAIS searches were above those of the high recall 
BBS search. The precision values for the WAIS searches were consistently below the results 
for the BBS high precision searches. To illustrate these differences, the distributions of 
judgments across the three categories for each of the four types of searches were compared. 
Since several cells in the matrix of values for each question contained zeroes or small values, 
the results for the four questions were aggregated for the Chi Square test. A Chi Square value 
of 1267.3 (p < .000, 6 DF) demonstrates the expected result that the distributions are 
independent. Thus, the proportions of relevant documents for the four types of search were 
different, i.e., these four types of search represent different search strategies when taken as a 
group (comparison of the two WAIS strategies are presented in a subsequent section and 
indicate that these strategies do not differ). 

Several observations can be made from these data. First, the high recall searches done in the 
BBS retrieved a higher number of relevant documents than the WAIS searches for each of the 
four questions. This is not surprising since the recall searches retrieved 241, 245, 490, and 
804 documents respectively for the four questions and the WAIS searches could return at 
most 45 documents. The consequence of this is that if users wish to obtain high recall and 
they are willing to spend the time examining large sets of citations, then a BBS should be 
strongly considered. 

Second, the high precision searches done in the BBS provided better precision results, 
although in two of the four questions the WAIS searches returned a higher number of relevant 
items. The implications of these data are less clear but suggest that if users are unwilling to 
examine even moderately sized document sets, better results may be obtained with a BBS 
search. 

3.1.2. Cases 

There were citations retrieved by both systems that were treated differently by the other 
system. Two of these instances are examined below. They illustrate citations which were 
retrieved by one system and given high relevance ratings but which were not retrieved by the 
other system. 

Case Ji Retrieved by Boolean-based System. The first case is that of a citation retrieved by 
the BBS but not by WAIS. Fourteen search terms that were in the WAIS verbose search 
strategy appear in the citation record. It seems likely, therefore, that the citation was identified 
by the WAIS system as a candidate for retrieval, but it was not included because it was not in 
the top 45 citations. The average frequency of occurrence of the terms is two. The terms in 
the search statement that occurred most frequently in the citation were "finite" and "element." 

It is possible that these two terms are pervasive enough in the NASA database that their 
weight is relatively low. 

It is important to note that there was no truncation capability in the WAIS implementation 
used in this study. Only the singular forms of the terms "error" and "grid" were entered in the 
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search statement, while the plural of each term appears in the citation. Therefore, the terms 
were not matched by the WAIS system. The lack of this functionality poses a major problem 
for users; in order to compensate, the user must include both singular and plural forms of all 
terms, as applicable. 

Case 2l Retrieved bv WAIS. The second case is for a citation retrieved by WAIS but not by 
the BBS. This citation has terms which match concepts used in the search statement used by 
the Boolean-based system. For example, one search statement was matched by the major 
descriptor "computational grids." Similar matches were found for four other search terms. 
However, the citation also contains a term that was eliminated from the search statement. The 
phrase "computational fluid dynamics" was removed from the final search set of the Boolean- 
based search by using the NOT operator because the set of retrieved citations was extremely 
large. The term was included in the WAIS search, where set size was not an issue. In 
retrospect, perhaps some other strategy to limit the set size should have been taken rather than 
eliminating a search term that retrieved a highly relevant document. 

This case is a good example of the pitfalls that confront users and trained searchers in 
working with a traditional retrieval system in a very large database. From an information 
point of view, removing the phrase from the final search set was not a good strategy because 
it eliminated this and, perhaps, other relevant citations. On the other hand, from a practical 
point of view, a huge citation set creates problems for the user. Even the limited set which 
excluded this citation contained more than 800 items; without this limitation, the set would 
have contained approximately 2500 items. Not many users have enough time to consider each 
citation in such a large set. 

3.2. Analysis of WAIS Searches 
3.2.1. Effect of Citation Set Size 

As described above, a WAIS system presents the user with a limited set of citations ordered 
by a score which is assumed to reflect relevance to the search statement, with the citation 
with the highest score at the top of the list. Therefore, the top part of a citation list should be 
richer in relevant citations than the bottom part. 

To explore the distribution of relevant documents in the WAIS ranked document lists, the 
lists were subdivided into three parts: top, middle, and bottom third of the ranked list. 
Precision values for these three sets of documents were then computed to determine if there 
was a higher concentration of relevant documents in the top-ranked set of results. Table 2 
presents the precision values for each question. Contrary to expectations that the ranking 
algorithm should distribute proportionally more relevant documents near the top of the 
rankings, no apparent pattern of distribution was found. 
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Table 2. Precision Values for Three Partitions of Ranked Results by Question. 


QUESTION 

TERSE QUERY 

VERBOSE QUERY 

Question 1 

Top Third 

.53 

.27 

Middle Third 

.38 

.47 

Bottom Third 

.64 

.43 

Question 2 

Top Third 

.55 

.55 

Middle Third 

.36 

.60 

Bottom Third 

.42 

.46 

Question 3 

Top Third 

.27 

.25 

Middle Third 

.27 

.15 

Bottom Third 

.21 

.00 

Question 4 

Top Third 

.33 

.07 

Middle Third 

.07 

.20 

Bottom Third 

.20 

.00 


A Chi-Square test was used to test whether the distribution of documents by relevance 
differed among the three subsets. The data from the four questions were combined by type of 
WAIS search then divided into three subsets, by position. The subsets are the top set (first 15 
items), middle set (second 15 items), and bottom set (third 15 items). The results for the 
Terse and Verbose searches are shown in Table 3. 


Table 3. Chi-Square Values for Distribution of Relevant Citations by Position in the Set 


SEARCH 

VALUE 

DF 

SIGNIFICANCE 

Terse 

1.88 

4 

.757 

Verbose 

2.59 

4 

.628 
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The results illustrated that for both types of query relevant documents were just as likely to 
occur in the middle and bottom thirds as in the top third of the retrieved document set. 


Various WAIS clients offer different cutoff levels; the Macintosh client used here offered a 
range from 5 to 50 citations. These results suggest that increasing the cutoff level for a WAIS 
search increases proportionally the relevant citations retrieved at least up to a citation set of 
45 items. Conversely, limiting the size of the retrieved set is likely to limit proportionally the 
number of relevant citations retrieved. This finding raises questions about the relationship 
between the WAIS score and the actual relevance of a citation as judged by the user. Further 
analysis of the relationship between relevance judgment and WAIS score is necessary. 

In sum, the ranking of documents in the WAIS searches did not distribute relevant documents 
in decreasing order. There was just as good a chance to find a relevant citation in the bottom 
third of the 45 retrieved as in the top third. Since the concentrations of relevant documents 
were good throughout the sets of 45, it seems prudent to provide higher cutoff values for 
users to choose if they wish. 

3.2.2. Effect of Differences in WAIS Search Strategies. Two types of search statements 
were used for the WAIS searches: terse and verbose (included synonyms and related terms). 
Chi-Square tests were run on each pair of results to determine whether or not the results of 
the searches differed in the distribution of citations by relevance. For Question 2, an expected 
value of less than five was obtained for one-third of the cells; therefore, the analysis for this 
question was discarded. The Chi-Square values for the other three questions are shown in 
Table 4. 


Table 4. Chi-Square Values for Distribution of Relevance Judgments 
by Type of WAIS Query Statement: Terse vs. Verbose. 


QUESTION 

VALUE 

DF 

SIGNIFICANCE 

1 

1.41 

2 

.494 j 

3 

15.34 

2 

.004 

4 

17.14 

2 

.000 


The Chi-Square value for Question 1 indicates that results of the two WAIS query strategies 
do not differ. That is, the results of the Terse search and the Verbose search were similar in 
the distribution of citations by relevance ratings. The results for Questions 3 and 4, however, 
indicate that there were differences between the results of the searches for each of these 
questions; however, they are in opposite directions. For question 3, the precision of the terse 
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queiy was lower than the precision of the verbose query; and for question 4, the precision of 
the Terse search was higher than for the Verbose search. Thus, there are no clear indications 
favoring either of the two strategies for these data. 

This result is somewhat surprising because the Verbose search included more terms and 
should, therefore, have been more representative of the full scope of concepts in the original 
query. However, it may be that expanding the search statement can dilute the precision of a 
WAIS search by introducing terms that are less closely related to the core of the query, i.e., 
more noise. In this implementation of WAIS, it is not possible to weight the search terms or 
to attach any measure of importance, such as order in the search statement— the WAIS 
algorithm weights all terms equally. A citation will be retrieved by the total score derived by 
the algorithm, even if a significant part of the score is contributed by terms that are less 
important to the query. 

3.2.2.I. Effects of Adding Terms 

To explore the length of query effects in more detail, WAIS searches were systematically 
conducted by adding terms one at a time. Figures 2-5 present graphical views of how the 
number of relevant documents changes as an additional term is added to a WAIS search. The 
number of relevant documents in the set of 45 retrieved is plotted as each term is added. 
Thus, for Figure 2, there were 8 relevant documents for the query consisting of only the term 
“unsteady,” six relevant documents for the query “unsteady aerodynamics,” and 1 2 relevant 
documents for the query “unsteady aerodynamics stall.” Frequency of occurrence of the term 
in thousands is given below each term. 

Question 1 was motivated by a somewhat general topic, and the terms and question evolved 
over the two reference interviews. The dips in the graph (fewer relevant citations among the 
45 retrieved) occurred as the frequently occurring and generic (for this database) terms 
“aerodynamics”(5500 occurrences in 4093 documents) and “air” (24,974 occurrences in 
14,049 documents) were added to the query. The most common term “model” (63,547 
occurrences in 34,027 documents) added some value since it is part of what may be 
considered a facet “stall delay model.” 

In question 2, the frequently occurring term “control” (76,573 occurrences in 27,170 
documents) adds value to “flight” (37,273 occurrences in 18,684 documents), but the third 
term of that facet “system” (87,826 occurrences in 44,492 documents) is too common and 
increases noise. The next facet, "modem control theory,” adds value word-by-word to reach a 
level almost as good as the simple “flight control” facet. Adding the new facet “flexible 
aircraft” adds surprisingly little value but does reinforce the need for using both terms of the 
facet. 

Question 3 illustrates the importance of technical terms in this database. The term “grid” 
(7,513 occurrences in 3,605 documents) is the most salient term, adding most of the relevant 
documents for the entire search. The term “mesh” (2,310 occurrences in 1,344 documents), 
although relatively rare, adds considerable noise, perhaps due to its multiple technical usages. 
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WAIS Search - Question 3 
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Figure 5 


WAIS Search - Question 4 
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Little value is added as additional generic terms are added until the enormously frequently 
occurring but technically important term “flow” (95,802 occurrences in 28,340 documents) is 
added to the query. 

Question 4 illustrates how a common facet, “computational fluid dynamics,” (computational 
occurs 20,767 times in 14,461 documents; fluid occurs 22,250 times in 13,977 documents; 
and dynamics occurs 27,051 times in 19,588 documents) is not helpful but the specific facet, 
“bluff body ,” (bluff occurs 373 times in 191 documents; body occurs 10,652 times in 6, 412 
documents) does yield some positive results. Examining ratios of total occurrences to the 
number of documents having occurrences is suggestive. Recall may be improved when these 
ratios are relatively high (e.g., model, control, grid, and flow improve recall and have ratios 
of 2 or higher) and the occurrences are fairly high. It may be useful to examine whether such 
ratios are useful for query formulation (since these could be provided to the user during query 
formulation automatically) and whether some tolerance levels for usefulness can be 
developed. 

This discussion illustrates the importance of mapping user need to facets and then considering 
which terms of a facet should actually be included in the query. This is certainly the case in 
BBS search strategy, but the use of controlled vocabularies makes these mapping decisions 
more tractable. For a system like WAIS, where the interplay between terms, facets, and the 
content of the entire database is so complex, the end-users special domain knowledge and 
immediately available intelligence seem necessary to give appropriate results. The end user’s 
understanding of the information problem, the vocabulary of the field, and the scope of the 
database may substitute for query formulation tools such as thesauri. 

3.2.3. Effect of Relevance Feedback. WAIS allows a user to use citations retrieved in one 
search as feedback for subsequent searches. The user marks citations and asks for other 
citations that are similar. Relevance feedback should re-sort the citations to bring more 
relevant items to the top of the ranked list, thus increasing both the recall and the precision 
within the cutoff value. 

Relevance feedback was used with each of the eight WAIS searches. In each case, the 
documents designated as examples for the new iteration were the five documents with the 
highest WAIS scores that had been rated as "Very Relevant" by the requestor. (In the WAIS 
search which returned only four documents rated as "Very Relevant," only four documents 
were designated.) The results of the new search were compared with the results of the original 
WAIS search. It is important to iterate that this was a simplistic approach to relevance 
feedback. First, several different approaches should be taken (e.g., how many documents to 
feedback, feedback of document sections or words as well as citations, etc.). Second, 
relevance feedback is best done by end users, a condition made impossible by the artificial 
constraints of this study. 

In six of eight searches, the results using relevance feedback were the same as the results 
from the original search. There were no differences in the specific citations retrieved nor in 
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the order of presentation of the citations. However, the results for two searches, both for 

Question 4, were different. The distribution of relevant citations for these two searches is 
shown in Table 5. 


Table 5. Relevance Judgments for Searches with Relevance Feedback for Question 4. 


SEARCH 

NOT 

RELEVANT 

SOMEWHAT 

RELEVANT 

VERY 

RELEVANT 

NO. 

% 

NO. 

% 

NO. 

% 

Terse with Relevance Feedback 

29 

(64%) 

7 

(16%) 

9 

(20%) 

Verbose with Relevance 
Feedback 

28 

(62%) 

15 

(33%) 

2 

( 4%) 


A Chi-Square test was used to test the expectation that the results of the searches using 
relevance feedback differed from the original WAIS searches. For the Verbose search an 
expected cell frequency of less than five was found for two of the six cells; therefore,’ the 
results are not reported. The results for the Terse search yield a value of .40 [p=.82]. Thus 
there are no significant differences between the original WAIS search and the search using’ 
relevance feedback. 


However, other comparisons are of interest. The search with relevance feedback for the Terse 
search yielded six new citations out of a total retrieved set of 45 citations. None of the new 
citations was judged as "Very Relevant." Therefore, relevance feedback did not increase 
recall. The precision value for the Terse search with relevance feedback is .20, which is 
approximately the same as the precision value for the original Terse search, .22. The Verbose 
search with relevance feedback yielded 20 new citations, none of which was judged "Very 
Relevant." The precision value for the Verbose search with relevance feedback is .04, which 
is much lower than the precision value for the original Verbose search, .09. 

The use of relevance feedback did not increase recall or precision in either of the two 
searches, although this result must be qualified by the small number of questions tested and 
the use of four or five citations as a whole as feedback. More extensive analyses of relevance 
feedback are needed to determine when and how it is most useful. 

4. Summary and Discussion 
4.1. Limitations of This Work 

Although simple Boolean-based retrieval systems (BBS) have been the standard in operational 
systems for many years, today’s systems have been enhanced to provide capabilities such as 
field limits, proximity qualification, and term weighting in the form of major or minor index 
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terms. Term weighting and the new rank or target features of Dialog were not used; therefore, 
the Dialog BBS system was not fully exploited. Likewise, the WAIS system was not fully 
exploited. One of the most basic features of WAIS is its ability to distribute queries to 
multiple servers; this feature was not used at all. Likewise, the highly promoted “natural 
language” feature was not used; strings of terms and phrases (structured queries) were used 
rather than full-sentence queries or English clauses. Since both systems are evolving, the 
results from this study must be considered in the context of the current state of the art. This is 
particularly crucial for the WAIS system that has a relatively small experience base-a fact 
that makes explorations such as this one more important since findings can be used to 
influence subsequent iterations. 

In addition to these limits on how the systems were used, this study was limited by the small 
number of questions, the fact that relevance judgments were made only by the requestor 
rather than a panel of experts, and the often-noted problems of making relevance judgments 
in general. 

As stated earlier, the objective of this investigation was to compare the capabilities of 
Boolean and vector-based retrieval systems, emphasizing the strengths and appropriate uses of 
each. The preliminary results of this investigation and the experience gained in working with 
both systems on the same questions provide information useful in reaching this goal and point 
out questions needing further study. Certain strengths and weaknesses of each system can be 
discussed within the context of these findings, with particular emphasis given to WAIS. 

4.2. Pros and Cons of BBS. 

BBSs offer powerful search capabilities for professional searchers. The main strengths are 
related to high levels of control over how search is conducted. The system can yield high 
recall values if high recall strategies are used, and high precision if high precision strategies 
are used. Thus, BBSs provide a good range of features for constructing searches customized 
to specific user needs. Features such as proximity operators and file limits give the searcher 
greater control in constructing a search query and allow results to be interpreted directly. 
Additionally, there is a large base of experience in using this type of system and extensive 
research on various search techniques and strategies. 

Limitations of BBS include the “flip side” of multiple features and good control-high levels 
of expertise are required. Searches require a considerable investment of proximity 
resources-time, effort, and money. The searchers in this project spent an average of three 
hours preparing for each search. Each search consisted of as many as 56 search statements 
and required at least an hour in an online session. Additionally, the size of the retrieved set 
can be very large, particularly when trying to maximize recall. The case cited earlier points 
out a trade-off encountered in trying to limit set size. 

4.3. Problems with WAIS 

This study illustrated several problems with the WAIS system. The lack of tools, such as 
proximity operators other than immediate adjoining to the right, means that the user has less 
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control over a search statement. Use of relevance feedback should be a compensating feature, 
but the results from relevance feedback were not encouraging. Secondly, the established size 
of the citation set artificially limits the number of relevant citations retrieved. Thirdly, the 
experience base, documentation, and training for vector-based systems are undeveloped, at 
this time. Some of these problems are likely due to the limited nature of the investigation and 
problems with server or client implementations, but others require longer-term consideration. 
First, WAIS provides a “black box” effect for the user. Users put in queries and get results 
based upon many hidden computations. For end users this may be just what is needed if 
performance can be demonstrated to be good. For professional intermediaries, however, using 
WAIS can be quite frustrating since it is impossible to deduce exactly how results were 
obtained. Furthermore, searchers have good control over how they conduct searches in a BBS; 
e.g., they can design queries to maximize either precision or recall. Strategies for constructing 
queries, setting cutoffs and using relevance feedback with WAIS are yet to be identified and 
tested for different types of needs. 

Second, the ranking algorithm used in WAIS yielded disparate results in this study. 

Substantial additions and modifications may be needed if WAIS is to be used in 
high-performance environments where results are mission-critical. This has been recognized 
by WAIS Inc. since the commercial version of the server provides for Boolean operators and 
a query report that gives posting data. Continued movement toward a more hybrid approach 
to information retrieval seems prudent if WAIS is to be useful for end users and professional 
intermediaries alike. More problematic is the way that relevance feedback is handled. At 
present, users have no control over how weights are assigned-a double-edged sword from 
Ieamability and user control perspectives. The low weights given to terms in the feedback 
query compared to terms in the original queiy reduced the effectiveness of relevance feedback 
in query refinement. Likewise, there is very poor control for setting cutoff values which 
affects both relevance judgments for ranked sets and feedback decisions. Also, distinctions 
between different iterations of a search are difficult to make unless each iteration is saved and 
then compared separately with a word processing or text management package. Adding new 
features for setting feedback parameters and for manipulating intermediate result sets will 
occupy the research and development community for some time to come. 

4.4. Potentials with WAIS 

Several positive aspects of using WAIS were illustrated by this work. First, the system yields 
generally acceptable precision values for little user effort. Second, less time and effort are 
required to conduct a search in WAIS than in the BBS. The time required to conduct a WAIS 
search ranged from approximately 40 seconds to slightly over 60 seconds. It must be noted, 
of course, that the preliminary work of structuring the search had been done for the BBS 
searches and was not counted as part of the WAIS search process. Also, WAIS utilizes 
natural language expressions as search queries, which should minimize the time needed to 
structure search statements. Third, the established size of the citation set provides a means of 
controlling the number of document records that the requestor must review, which is valuable 
in producing results that are immediately useful. 
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Although it is sometimes the case that no documents are returned in a WAIS search, it is 
much more likely that something is found that can serve as a starting point for subsequent 
search or browsing. This is in contrast to BBS systems such as OPACs and Grateful Med that 
typically yield “no hits” for about one-third of all end user searches. Thus, WAIS offers users 
starting points for interactive sessions. Although the purpose of this study centered on 
retrieval performance rather than the interface, usability effects were clearly apparent and 
represent significant advantages over BBS. First, there is no query language to learn— a clear 
advantage for novice and occasional users. Second, WAIS is easy to use, offering direct 
manipulation interaction styles in the Macintosh and X-Windows clients. These clients invite 
exploration and interaction and allow editing/cutting and pasting operations to be leveraged as 
reinforcement to continuous interactivity rather than as simple short-cuts for discrete, linear 
search. 

4.5. Implications 

Highly interactive systems invite end users to do their own searching and offer massive 
amounts of information. Although much time can be spent exploring with WAIS (other 
systems such as Mosaic depends almost entirely on exploration), questions about efficiency 
and cost-effectiveness will become more critical as these systems are used in commercial 
settings and as the costs of network access and use become less transparent. 

There are two sets of problems that require research. First, it seems clear that highly 
interactive front-ends such as WAIS demand new information-seeking strategies. Just as it 
took time and experience for the online community to develop and teach strategies for BBS 
such as “building block,” “successive fraction,” “pearl growing,” and “iterative scanning” 
(Hawkins & Wager, 1982), it will take time to discover strategies that serve different search 
purposes and users in WAIS and other emerging systems. For example: 

• What effects will query length have in such environments, and how should users 

control relevance feedback? 

• Will domain experts use relevance feedback differently than intermediaries? 

• Should term weighting be used, and if so, how? 

• How will multiple servers affect results? 

Just as intermediaries have developed divide and conquer strategies to handle problems with 
many facets using a BBS, there may be a need for staged strategies for different databases or 
customization of queries for specific databases using WAIS. Although WAIS clients provide 
options to display results by server, this is a simple beginning to distinguish and develop 
more carefully targeted revisions/feedback to specific servers. Finally, better ways are needed 
to keep the user informed of how the search is progressing and why results have been 
obtained. Intermediate posting data and sophisticated query reports may be helpful here. 

Second, there is a need for new evaluation metrics that address the interactive nature of 
systems like WAIS. Variables such as time, number of queries, number of iterations, number 
of screens/windows displayed, number or distribution of records displayed or saved, cost, and 
satisfaction may take on new meanings in such environments. New variables such as number 
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of servers queried, number of servers returning results, and filtering values for different 
servers may be needed. As electronic environments become more seamlessly integrated, it 
may become feasible to trace how information actually gets integrated into user work (e.g., 
text or citations cut and pasted into documents, figures imported into spreadsheets or local 
databases, etc.) to gain a fuller understanding of how information seeking supports user work. 
It will become more feasible to assess where information objects came from as electronic 
documents are constructed as a series of links between existing and original text, images, and 
other objects. Some attention to assessing the overall process rather than simply the products 
of searching must also be considered (e.g., search patterns from symbolic or graphical 
perspectives, Lin, Liebscher, & Marchionini, 1991). 

The results summarized here demonstrate some of the possibilities as well as problems that 
systems such as WAIS provide for the IR community. Rather than aiming to compare systems 
on simplistic metrics rooted in tradition, we are challenged to develop systems that apply 
different techniques and interfaces and to test those systems from formative as well as 
comparative perspectives. This effort used comparison as the basis for better understanding a 
specific innovative system. The results temper the “hype" associated with new technologies 
and raise questions about assumed performance and features. It suggests directions for 
improving this particular system in subsequent iterations, but more importantly, raises issues 
about what types of questions seem promising for research, development, and evaluation. 
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Subject One 

DIALOG Recall Search Statement: 

File 108:AEROSPACE 62-93/9306B2 
Set Items Description 


?ss unstead? 

SI 20326 UNSTEAD? 


Search the unsteady state concept 


?ss si ()(state? or flow? or aerodynam? or pressure?) 

20326 SI 

52 153877 STATE? 

53 247128 FLOW? 

54 58099 AERODYNAM? 

55 150226 PRESSUR? 

56 14334 SI ()(STATE? OR FLOW? OR AERODYNAM? OR PRESSUR?) 

?ss turbulen? 

57 70594 TURBULEN? 


?ss s7()(flow? or wake? or model? or effect?) 

70594 S7 
247128 S3 

58 12835 WAKE? 

59 335200 MODEL? 

510 455590 EFFECT? 

511 29748 S7()(S3 OR WAKE? OR MODEL? OR EFFECT?) 

?ss s7()boundar?()layer? 

70594 S7 

SI 2 135730 BOUNDAR? 

513 121839 LAYER? 

514 11509 S7()BOUNDAR?() LAYER? 

?ss nonstation? 

SI 5 4758 NONSTATION? 


?ss non()station? 

SI 6 37066 NON 

51 7 66489 STATION? 

51 8 525 NON()STATION? 

?ss si 5 or si 8 

4758 SI 5 
525 SI 8 

SI 9 5108 S15 0RS18 
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?SS Si 9()s7 

5108 SI 9 
70594 S7 
S20 34 S19()S7 

?ss si or s5 or s6 or s7 or sll or s14 or s19 or s20 

Combine synonyms for the unsteady state 

concept 

20326 SI 
150226 S5 
14334 S6 
70594 S7 
29748 S11 
11509 SI 4 
5108 SI 9 
34 S20 

521 223959 Si OR S5 OR S6 OR S7 OR S1 1 OR SI 4 OR SI 9 OR 

S20 

?ss s21 or (model?()air()flow?) or (airQflow?) 

223959 S21 

522 335200 MODEL? 

523 115109 AIR 

524 247128 FLOW? 

525 1 MODEL?(W)AIR(W)FLOW? 

526 115109 AIR 

527 247128 FLOW? 

528 7465 AIR(W)FLOW? 

529 227922 S21 OR (MODEL?()AIR()FLOW?) OR (AIR()FLOW?) 

?ss s29/de* 

Sift to see how many citations were pulled 
from major descriptors 

530 83616 S29/DE* 

?ss s29/de 

Sift to see how many citations were pulled 
from all descriptors 

531 164207 S29/DE 
?ss model? 

Search the mathematical/computer models 

concept 

532 335200 MODEL? 

?ss math?()s32 

533 166591 MATH? 

335200 S32 

534 87641 MATH?()S32 
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?ss computer?()s32 

535 199247 COMPUTER? 

335200 S32 

536 4145 COMPUTER?()S32 

?ss (numeric?(2n)s35)()s32 

537 131256 NUMERIC? 

199247 S35 
335200 S32 

538 34 (NUMERIC?(2N)S35)()S32 

?ss digit?()simulat? 

539 61786 DIGIT? 

540 126899 SIMULAT? 

541 5918 DIGIT? OSIMULAT? 

?ss s32 or s34 or s36 or s38 or s40 or s41 

Combine synonyms for the mathematical/ 
computer models concept 

335200 S32 
87641 S34 
4145 S36 
34 S38 
126899 S40 
5918 S41 

S42 408079 S32 OR S34 OR S36 OR S38 OR S40 OR S41 
?ss s42/de* 


S43 109315 S42/DE* 
?ss s42/de 


S44 237530 S42/DE 


?ss propell? 

S45 29531 PROPELL? 


Sift to see how many citations were pulled 
from major descriptors 


Sift to see how many citations were pulled 
from all descriptors 


Search the propeller state concept 


?ss high()solid?()s45 

546 289326 HIGH 

547 86661 SOLID? 

29531 S45 

548 2 HIGH()SOLID?()S45 
?ss s45()blade? 
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29531 S45 

549 23717 BLADE? 

550 1027 S45()BLADE? 

?ss (compress? or fan? or rotor? or stator or turbine?)()s49 

551 54002 COMPRESS? 

552 6742 FAN? 

553 22141 ROTOR? 

554 1985 STATOR 

555 30977 TURBINE? 

23717 S49 

S56 15931 (COMPRESS? OR FAN? OR ROTOR? OR STATOR OR 
TURBINE?)()S49 


?ss wind()s55 

557 85351 WIND 
30977 S55 

558 2248 WIND()S55 

?ss (s57()power?)()(utiliz? or generat?) 

85351 S57 

559 146391 POWER? 

560 63826 UTILIZ? 

561 121687 GENERAT? 

562 89 (S57() POWER?) ()(UTILIZ? OR GENERAT?) 

?ss streamline?()bod? 

563 4040 STREAMLIN? 

564 93722 BOD? 

565 275 STREAMLIN?()BOD? 

?ss engine()part? 

566 47311 ENGINE 

567 296392 PART? 

568 3034 ENGINEQPART? 

?ss windmill? 

569 886 WINDMILL? 


?ss s45 or s48 or s49 or s50 or s56 or s58 or s62 or s65 or s68 
or s69 or s55 


Combine synonyms lor the propeller concept 

29531 S45 
2 S48 
23717 S49 
1027 S50 
15931 S56 
2248 S58 
89 S62 
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275 S65 
3034 S68 
886 S69 
30977 S55 

S70 72042 S45 OR S48 OR S49 OR S50 OR S56 OR S58 OR S62 
OR S65 OR S68 OR S69 OR S55 


?ss s70/de* 


S71 43374 S70/DE* 

?ss s70/de 


572 59245 S70/DE 
?ss delay? 

573 18147 DELAY? 

?ss stall?()s73 

574 4056 STALL? 

18147 S73 

575 25 STALL?()S73 


Sift to see how many citations were pulled 
from major descriptors 


Sift to see how many citations were pulled 
from all descriptors 


Search the stall delay concept 


?ss aerodynam?()(s74 or stabil?) 

576 58099 AERODYNAM? 

4056 S74 

577 135757 STABIL? 

578 4726 AERODYNAM?()(S74 OR STABIL?) 

?ss engine?()fail? 

579 155807 ENGINE? 

580 45414 FAIL? 

581 1209 ENGINE?()FAIL? 


?ss (aircraft? or airplane?)()perform? 

582 121142 AIRCRAFT? 

583 6021 AIRPLANE? 

584 228951 PERFORM 

585 5671 (AIRCRAFT? OR AIRPLANE??)()PERFORM? 

?ss stop???? ? 

586 5945 STOP???? ? 


?ss lateness 

S87 22 LATENESS 
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?ss time()lag? 

588 251319 TIME? 

589 20890 LAG? 

590 7038 TIME?()LAG? 


?ss s73 or s74 or s75 or s77 or s78 or s81 or s85 or s86 or s87 
or s90 

Combine synonyms for the stall delay concept 

18147 S73 
4056 S74 
25 S75 
135757 S77 
4726 S78 
1209 S81 
5671 S85 
5945 S86 
22 S87 
7038 S90 

S91 168912 S73 OR S74 OR S75 OR S77 OR S78 OR S81 OR S85 
OR S86 OR S87 OR S90 


?ss s91/de* 

Sift to see how many citations were pulled 
from major descriptors 

592 73227 S91/DE* 

?ss s91/de 

Sift to see how many citations were pulled 
from all descriptors 

593 121954 S91/DE 


?ss s30 and s43 and s71 and s92 

Combine the four separate concepts together 
(unsteady state, mathematical/computer 
models, propeller, and stall delay) for the 
sets with major descriptor terms 

83616 S30 
109315 S43 
43374 S71 
73227 S92 

S94 1 1 S30 AND S43 AND S71 AND S92 


?ss s31 and s44 and s72 and s93 


164207 S31 
237530 S44 


Combine the four separate concepts together 
(unsteady state, mathematical/computer 
models, propeller, and stall delay) for the 
sets with all descriptor terms 
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59245 S72 
121954 S93 

S95 243 S31 AND S44 AND S72 AND S93 


?ss s29 and s42 and s70 and s91 

Combine the four separate concepts together 
(unsteady state, mathematical/computer 
models, propeller, and stall delay) for the 
sets with natural language terms 

227922 S29 
408079 S42 
72042 S70 
168912 S91 

S96 1334 S29 AND S42 AND S70 AND S91 


?ss ud-9001 :9212b2 

S97 205080 UD=9001 :9212B2 


Set the update restriction of 1990-1992 


?ss s94 and s97 

11 S94 
205080 S97 

S98 3 S94ANDS97 


Limit the major descriptor citations by date 


?ss s95 and s97 

243 S95 
205080 S97 

S99 48 S95ANDS97 


Limit all descriptor citations by date 


?ss s96 and s97 

1334 S96 
205080 S97 

SI 00 244 S96ANDS97 


Limit the natural language citations by date 


?ss s99 not s98 

Compared sets by NOTing to compare 
citations pulled in through major descriptors, 
all descriptors, and natural language 

48 S99 
3 S98 

SI 01 45 S99 NOT S98 

?ss si 00 not (s98 or si 01) 

244 SI 00 
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3 S98 
45 S101 

Si 04 196 SI 00 NOT (S98 OR SI 01) 

?logoff 



Subject Two 

DIALOG Recall Search Statement: 

File 108:Aerospace Database 1 962-1 993/Jul B1 


Set Items Description 


Expand on select authors' names and titles to 
look at indexing 


51 10 AU-"MYERS, THOMAS T." 

52 20 AU*MCRUER, D. 

53 47 AU=MCRUER, D. T. 

54 5 AU=MCRUER, DUANE 

55 16 AU=MCRUER, DUANE T. 

56 88 E4-E7 

57 2 AU-THOMSON, P. F. 

58 2 AU=THOMSON, P.J. 

59 4 E4-E5 

510 2 AU=ASHKENAS, I. 

511 42 AU=ASHKENAS, I. L. 

51 2 4 AU=ASHKENAS, IRVING L. 

SI 3 48 E6-E8 

514 16 AU=DOYLE, J. 

51 5 32 AU-DOYLE, J. C. 

SI 6 48 E4-E5 

SI 7 15649 ADVANCE7/TI 
SI 8 6969 PILOT7/TI 

51 9 40127 AIRCRAFT™ 

520 30490 FLIGHT™ 

521 61210 CONTROL™ 

S22181326 SYSTEM?™ 

523 2 ADVANCE?()PILOT?()AIRCRAFT()FLIGHT()CONTROL() 

SYSTEM?/TI 

524 2 RN=STI-TR-1 228-1 


?ss modern()control?()theor? 


Search the modern control theory concept 


525 9189 MODERN 

526 235735 CONTROL? 

527 304734 THEOR? 

528 239 MODERN()CONTROL?()THEOR? 

?ss met 

529 77 MCT 


?ss controlQtheor? 
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530 235735 CONTROL? 

531 304734 THEOR? 

532 13852 CONTROL?()THEOR? 


?e h-in 


Ref Items Index-term 
El 8 H-BETA LINE 

E2 4 H-GAMMA LINE 

E3 0 "H-IN 

E4 53 H-INFINITY CONTROL 

E5 59 H-LINE 

E6 53 H-WAVE 

E7 10 H-1 ENGINE 

E8 7 H-1 ROCKET ENGINE 

E9 3 H-1 26 AIRCRAFT 

E10 1 H-126 JET FLAP AIRCRAFT 

Ell 1 H-13 AIRCRAFT 

E12 6 H-1 7 HELICOPTER 


Expand on h-infinity control in the basic 
index 


?ss e4 

S33 53 "H-INFINITY CONTROL" 


Select the appropriate term 


?ss h()infinit?()control? 

534 40267 H 

535 23343 INFINIT? 

536 235735 CONTROL? 

537 247 H()INFINIT?()CONTROL? 


?ss h ()infinit? 

538 40267 H 

539 23343 INFINIT? 

540 635 H()INFINIT? 


?e h-2 

Ref Items Index-term 
El 6 H-1 7 HELICOPTER 

E2 1 H-1 9 HELICOPTER 

E3 0 "H-2 
E4 1 H-25 HELICOPTER 

E5 1 H-34 HELICOPTER 

E6 1 H-43 HELICOPTER 

E7 82 H-53 HELICOPTER 

E8 7 H-54 HELICOPTER 

E9 14 H-56 HELICOPTER 

E10 43 H-60 HELICOPTER 


Expand on h-2 control in the basic index 
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Ell 304 HA 
El 2 11 HAA 


?e h 2 

Ref Items Index-term 

El 13 H REGION 

E2 161 H WAVES 

E3 0 *H 2 
E4 63 H-ALPHA LINE 

E5 8 H-BETA LINE 

E6 4 H-GAMMA LINE 

E7 53 H-INFINITY CONTROL 

E8 59 H-LINE 

E9 53 H-WAVE 

E10 10 H-1 ENGINE 

Ell 7 H-1 ROCKET ENGINE 

E12 3 H-126 AIRCRAFT 

?ss h()2()control? 

541 40267 H 

542 195123 2 

543 235735 CONTROL? 

544 3 H()2()CONTROL? 

?ss s28 or s29 or s32 or s33 or s37 or s40 or s44 

Combine synonyms for the modern control 
theory concept 

239 S28 
77 S29 
13852 S32 
53 S33 
247 S37 
635 S40 
3 S44 

545 14284 S28 OR S29 OR S32 OR S33 OR S37 OR S40 OR S44 


?ss s45/de* 

546 7280 S45/DE* 
?ss s45/de 

547 12827 S45/DE 
?ss flight?()simulat? 

548 110968 FLIGHT? 


Sift through citations in which these 
concepts were major descriptors 


Sift through those citations in which these 
concepts were major or minor descriptors 


Searching the flight control concept 
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549 126899 SIMULAT? 

550 8968 FLIGHT?()SIMULAT? 

?ss flight?()control?()design? 

551 110968 FLIGHT? 

552 235735 CONTROL? 

553 260987 DESIGN? 

554 125 FLIGHT?()CONTROL?()DESIGN? 


?ss fligh1?()control?()theor? 

555 110968 FLIGHT? 

556 235735 CONTROL? 

557 304734 THEOR? 

558 5 FLIGHT?()CONTROL?()THEOR? 

?ss flight?()control? 

559 110968 FLIGHT? 

560 235735 CONTROL? 

561 10621 FLIGHT?()CONTROL? 

?ss active?()control? 

562 104513 ACTIV? 

563 235735 CONTROL? 

564 3531 ACTIV?()CONTROL? 

?ss act 

565 4525 ACT 

?ss activ?()control?()technol? 

566 104513 ACTIV? 

567 235735 CONTROL? 

568 108774 TECHNOL? 

569 231 ACTIV?()CONTROL?()TECHNOL? 

?ss activ?()control?()tech? 

570 104513 ACTIV? 

571 235735 CONTROL? 

572 344509 TECH? 

573 270 ACTIV?()CONTROL?()TECH? 

?ss flight?()control?()system? 

574 110968 FLIGHT? 

575 235735 CONTROL? 

576 509309 SYSTEM? 

577 2896 FLIGHT?()CONTROL?()SYSTEM? 

?ss control?()system?()design? 

578 235735 CONTROL? 
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579 509309 SYSTEM? 

580 260987 DESIGN? 

581 10793 CONTROL?()SYSTEM?()DESIGN? 

?ss aircraft?()control? 

582 121142 AIRCRAFT? 

583 235735 CONTROL? 

584 6658 AIRCRAFT?()CONTROL? 

?ss advanc?(2w)s82(advanc?(2w)s82)(5n)s54 

585 47210 ADVANC? 

121142 S82 

125 S54 

586 0 (ADVANC?(2W)S82)(5N)S54 

?ss (robust(5n)multivari?)(7n)(control?()method?) 

587 5109 ROBUST 

588 5523 MULTIVARI? 

589 235735 CONTROL? 

590 351482 METHOD? 

591 2 (ROBUST(5N)MULTIVARI?)(7N)(CONTROL?()METHOD?) 

?ss (single or singular?)(w)(value?(w)base?)(7n)s77 

592 86999 SINGLE 

593 14469 SINGULAR? 

594 145770 VALUE? 

595 216862 BASE? 

2896 S77 

596 2 (SINGLE OR SINGULAR?)(W)(VALUE?(W)BASE?)(7N)S77 

?ss computer?()simulat? 

597 199247 COMPUTER? 

598 126899 SIMULAT? 

599 43205 COMPUTER?()SIMULAT? 

?ss aircraft?()guid? 

51 00 121142 AIRCRAFT? 

5101 34575 GUID? 

51 02 1478 AIRCRAFT?()GUID? 

?ss stabiliz? 

SI 03 25301 STABILIZ? 

?ss s50 or s54 or s58 or s61 or s64 or s65 or s73 or s77 or s81 
or s84 

Combine synonyms for the flight control 
theory concept 

8968 S50 
125 S54 
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5 S58 
10621 S61 
3531 S64 
4525 S65 
270 S73 
2896 S77 
10793 S81 
6658 S84 

SI 04 38491 S50 OR S54 OR S58 OR S61 OR S64 OR S65 OR S73 
OR S77 OR S81 OR S84 

?ss si 04 or s82 or s91 or s96 or s99 or si 02 or si 03 

38491 S104 
121142 S82 
2 S91 
2 S96 
43205 S99 
1478 SI 02 
25301 SI 03 

SI 05 201581 SI 04 OR S82 OR S91 OR S96 OR S99 OR SI 02 OR 
SI 03 

?ss si 05/de* 

Sift through those citations in which these 
concepts were major descriptors 

SI 06 110943 Si 05/DE* 

?ss si 05/de 

Sift through those citations in which these 
concepts were major or minor descriptors 

SI 08 161600 SI 05/DE 


?ss (flex? or elast?)()(aircraft? or bod? or model? or airplane?) 

Search on the flexible aircraft concept 

51 09 27359 FLEX? 

51 10 71257 ELAST? 

5111 121142 AIRCRAFT? 

51 12 93722 BOD? 

51 13 335200 MODEL? 

51 14 6021 AIRPLANE? 

51 15 8589 (FLEX? OR ELAST?)()( AIRCRAFT? OR BOD? OR MODEL? 

OR AIRPLANE?) 


?ss s115()(dynamic? or design? or handl? or model?) 
8589 SI 15 

51 16 168075 DYNAMIC? 

51 17 260987 DESIGN? 

51 18 18085 HANDL? 
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51 19 335200 MODEL? 

51 20 405 SI 15()(DYNAMIC? OR DESIGN? OR HANDL? OR MODEL?) 

?ss s120()model? 

405 SI 20 

51 21 335200 MODEL? 

SI 22 354 S120()MODEL? 

?ss si 16()s1 21 

168075 S1 16 
335200 SI 21 

SI 23 10541 S1 1 6()S1 21 


?ss s116()system? 

168075 S1 16 
SI 24 509309 SYSTEM? 

Si 25 7219 S116()SYSTEM? 

?ss transport?()aircraft? 

SI 26 66796 TRANSPORT? 

SI 27 121142 AIRCRAFT? 

SI 28 6061 TRANSPORT?()AI RCRAFT? 

?ss s127()s121 

121142 S127 
335200 SI 21 

SI 29 2726 S127()S121 

?ss control?()configure?()vehicle? 

SI 30 235735 CONTROL? 

SI 31 78429 CONFIGUR? 

SI 32 53481 VEHICLE? 

51 33 600 CONTROL?()CONFIGUR?()VEHICLE? 

?ss activ?()control?()tech?()s1 27 

51 34 104513 ACTIV? 

51 35 235735 CONTROL? 

SI 36 344509 TECH? 

121142 Si 27 

SI 37 2 ACTI V?()CONTROL?()TECH?()Sl 27 


?ss aeroelastic? 

SI 38 5395 AEROELASTIC? 

?ss (structur? or si 27 or s130)()stabili? 

51 39 282772 STRUCTUR? 

121142 SI 27 
235735 SI 30 

51 40 135626 STABILI? 
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5141 15126 (STRUCTUR? OR S127 OR S130)()STABILI? 

?ss rigid?()structur?s1 39 

51 42 18975 RIGID? 

282772 SI 39 

5143 4296 RIGID?()S1 39 

?ss s127()configur? 

121142 Si 27 

51 44 78429 CONFIGUR? 

51 45 4300 S127()C0NFIGUR? 

?ss S115 or si 20 or s123 or s125 or s128 or s129 or s133 or s137 
or si 38 

Combine synonyms for the flexible aircraft 

concept 

8589 SI 15 
405 SI 20 
10541 SI 23 
7219 SI 25 
6061 SI 28 
2726 SI 29 
600 SI 33 
2 SI 37 
5395 Si 38 

Si 46 38615 S1 15 OR S120 OR S123 OR S125 OR S128 OR S129 OR 
Si 33 OR Si 37 OR SI 38 

?ss si 46 or si 41 or si 43 or si 45 
38615 SI 46 
15126 S141 
4296 SI 43 
4300 SI 45 

Si 47 58305 S146 OR S141 OR Si 43 OR S145 
?ss si 47/de* 

Sift through those citations in which these 
concepts were major descriptors 

SI 48 30559 Si 47/DE* 

?ss si 47/de 

Sift through those citations in which these 
concepts were major or minor descriptors 

SI 49 49829 Si 47/DE 

?ss s46 and si 06 and si 48 

Combine the separate concepts (modern 
control theory, flight control, and flexible 


44 


aircraft) for those citations in which the 
search terms were major descriptors 

7280 S46 
110943 SI 06 
30559 SI 48 

51 53 201 S46 AND Si 06 AND SI 48 
?ss s47 and si 08 and si 49 

Combine the separate concepts (modern 
control theory, flight control, and flexible 
aircraft) for those citations in which the 
search terms were minor descriptors 

12827 S47 
161600 SI 08 
49829 SI 49 

51 54 1135 S47 AND SI 08 AND SI 49 
?ss s45 and si 05 and si 47 

Combine the separate concepts (modern 
control theory, flight control, and flexible 
aircraft) for those citations in which the 
search terms were in natural language 

14284 S45 
201581 SI 05 
58305 SI 47 

51 55 1819 S45 AND SI 05 AND SI 47 
?ss ud=09001 :9212b2 

Set the update for 1990-1992 
SI 56 205080 UD=9001 :9212B2 

?ss si 53 and si 56 

Restrict the citations with major descriptors 
by update 

201 Si 53 
205080 SI 56 

SI 57 71 Si 53 AND Si 56 

?ss si 54 and si 56 

Restrict the citations with major or minor descriptors 
by update 

1135 SI 54 
205080 SI 56 

SI 58 357 SI 54 AND Si 56 


?ss si 55 and si 56 

Restrict the citations with natural language 
by update 
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1819 SI 55 
205080 SI 56 

SI 59 498 Si 55 AND SI 56 

?ss si 58 not si 57 

Compared sets by NOTing to compare 
citations pulled in through major descriptors, 
minor descriptors, and natural language 

357 SI 58 
71 SI 57 

SI 60 286 SI 58 NOT SI 57 

?ss si 59 not (si 57 or si 60) 

498 SI 59 
71 SI 57 
286 SI 60 

SI 61 141 SI 59 NOT (SI 57 OR Si 60) 

?logoff 
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Subject Three 


DIALOG Recall Search Statement: 
File 108:AEROSPACE 62-93/9306B2 

Set Items Description 


?ss computation?()grid? 

The computational gridding concept 

51 79544 COMPUTATION? 

52 21657 GRID? 

53 8475 COMPUTATION?()GRID? 

?ss adapt?()grid? 

54 33301 ADAPT? 

55 21657 GRID? 

56 470 ADAPT?()GRID? 

?ss mesh()redistrib? 

57 6208 MESH 

58 3424 REDISTRIB? 

59 2 MESH()REDISTRIB? 

?ss structur?()grid? 

510 281815 STRUCTUR? 

511 21657 GRID? 

512 154 STRUCTUR?()GRID? 

?ss grid?()generat? 

51 3 21657 GRID? 

514 121348 GENERAT? 

51 5 2688 GRID?()GENERAT? 

?ss computat?()fluid?()dynamic? 

SI 6 79574 COMPUTAT? 

SI 7 95669 FLUID? 

51 8 167348 DYNAMIC? 

519 25040 COMPUTAT?()FLUID?()DYNAMIC? 

?ss adapt?()control? 

520 33301 ADAPT? 

521 235043 CONTROL? 

522 8383 ADAPT?()CONTROL? 

?ss flow()distrib? 

523 235042 FLOW 

524 272625 DISTRIB? 

525 22463 FLOW()DISTRIB? 
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?ss s3 or s6 or s9 or si 2 or si 5 or si 9 or s22 or s25 

Combine similar terms for the computational 
gridding concept 

8475 S3 
470 S6 
2 S9 
154 S12 
2688 SI 5 
25040 SI 9 
8383 S22 
22463 S25 

526 56686 S3 OR S6 OR S9 OR SI 2 OR SI 5 OR SI 9 OR S22 OR S25 
?ss finite()differen?()theor? 

The finite difference and math theory 

concept 

527 76274 FINITE 

528 230475 DIFFEREN? 

529 304131 THEOR? 

530 15945 FINITE()DIFFEREN?()THEOR? 

?ss truncat?()error? 

531 4976 TRUNCAT? 

532 83056 ERROR? 

533 1658 TRUNCAT?()ERROR? 

?ss numeric?()analy? 

534 130784 NUMERIC? 

535 546777 ANALY? 

536 31774 NUMERIC?()ANALY? 

?ss math?()model? 

537 166198 MATH? 

538 333985 MODEL? 

539 87417 MATH?()MODEL? 

?ss coordinate? ? 

540 33623 COORDINATE? ? 

?ss coordinat? 

541 37270 COORDINAT? 

?ss problem?()solv? 

542 218104 PROBLEM? 

543 391479 SOL? 

544 10792 PROBLEM?()SOL? 

?ss precision? 
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S45 13757 PRECISION? 


?ss approximat? 

S46 106466 APPROXIMAT? 


?ss finite()element?()method? 

547 76274 FINITE 

548 117011 ELEMENT? 

549 350376 METHOD? 

550 29009 FINITE()ELEMENT?()METHOD? 

?ss s30 or s33 or s36 or s39 or s41 or s44 or s45 or s46 or s50 

Combine synonyms for the finite difference 
and math theory concept 

15945 S30 
1658 S33 
31774 S36 
87417 S39 
37270 S41 
10792 S44 
13757 S45 
106466 S46 
29009 S50 

S51 289158 S30 OR S33 OR S36 OR S39 OR S41 OR S44 OR S45 OR 
S46 OR S50 


?ss s26 and s51 

Combine the two main concepts 

56686 S26 
289158 S51 

552 20390 S26 AND S51 
?ss ud=9001 :921 2b2 

Set the update for 1900-1992 

553 205080 UD=9001:9212B2 

?ss s52 and s53 

Limit the final set by the update 

20390 S52 
205080 S53 

554 4447 S52 AND S53 
?ss s54/de* 

Sift through the final set to see how many 
citations were pulled from major descriptors 

S57 3679 S54/DE* 

?ss s54/de 

Sift through the final set to see how many 
citations were pulled from major or minor descriptors 
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S58 4433 S54/DE 


?ss s54 not s58 

Number of records in the set retrieved with 
only title and abstract words, i.e. not descriptors. 
See also set 68. 

4447 S54 
4433 S58 

559 14 S54 NOT S58 

?ss s59 not s57 

14 S59 
3679 S57 

560 14 S59 NOT S57 

?ss s54 not (s41 or s44 or s45 or s46) 

Remove terms that seem irrelevant to the 
query and caused major jumps in retrieval 

4447 S54 
37270 S41 
10792 S44 
13757 S45 
106466 S46 

561 2549 S54 NOT (S41 OR S44 OR S45 OR S46) 

?ss s61 not (si 9 or s25) 

2549 S61 
25040 SI 9 
22463 S25 

562 805 S61 NOT (SI 9 OR S25) 

?ss s61/de* 


S63 2146 S61/DE* 

?ss s61/de 


564 2546 S61/DE 
?ss s62/de* 

Sift through the final set to see how many 
citations were pulled from major descriptors 

565 643 S62/DE* 
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?ss s62/de 

Sift through the final set to see how many 
citations were pulled from major or minor descriptors 

S66 803 S62/DE 

?ss s66 not s65 


803 S66 Number of records pulled from minor descriptors only 

643 S65 

567 160 S66NOTS65 
?ss s62 not (s65 or s66) 

805 S62 Number of records retrieved with 

643 S65 only title and abstract words, i.e. not descriptors 

803 S66 

568 2 S62 NOT (S65 OR S66) 

?ss logoff 
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Subject Four 

DIALOG Recall Search Statement: 
File 108:AEROSPACE 62-93/9306B2 

Set Items Description 


?ss unstead? 

SI 20243 UNSTEAD? 


Search the unsteady/turbulent concept 


?ss unstead?()state? 

52 20243 UNSTEAD? 

53 153467 STATE? 

54 1478 UNSTEAD?()STATE? 

?ss computat?()fluid?()dynamic? 

55 79574 COMPUTAT? 

56 95669 FLUID? 

57 167348 DYNAMIC? 

58 25040 COMPUTAT?()FLUID?()DYNAMIC? 

?ss turbulen?()flow? 

59 70384 TURBULEN? 

510 246544 FLOW? 

511 21177 TURBULEN?()FLOW? 

?ss unstead?()flow? 

SI 2 20243 UNSTEAD? 

SI 3 246544 FLOW? 

S14 10785 UNSTEAD?()FLOW? 


?ss unstead?()aerodynamic? 

SI 5 20243 UNSTEAD? 

51 6 57908 AERODYNAMIC? 

51 7 2784 UNSTEAD?()AERODYNAMIC? 

?ss turbulen?()wake? 

SI 8 70384 TURBULEN? 

SI 9 12792 WAKE? 

520 2451 TURBULEN?()WAKE? 

?ss turbulen?()model? 

521 70384 TURBULEN? 

522 333985 MODEL? 

523 4584 TURBULEN?()MODEL? 


?ss turbulen? 
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524 70384 TURBULEN? 

?ss pressur? 

525 149936 PRESSUR? 

?ss unstead?()pressur? 

526 20243 UNSTEAD? 

527 149936 PRESSUR? 

528 734 UNSTEAD?()PRESSUR? 

?ss turbulen?()effect? 

529 70384 TURBULEN? 

530 454543 EFFECT? 

531 4812 TURBULEN?()EFFECT? 

?ss nonstation?()turbulen? 

532 4751 NONSTATION? 

533 70384 TURBULEN? 

534 29 NONSTATION?()TURBULEN? 

?ss nonstation? 

535 4751 NONSTATION? 

?ss turbulen?()boundar?()layer? 

536 70384 TURBULEN? 

537 135404 BOUNDAR? 

538 121533 LAYER? 

539 11472 TURBULEN?()BOUNDAR?()LAYER? 

?ss si or s4 or s8 or si 1 or si 4 or si 7 or s20 or s23 or s24 or 
s25 or s28 

Combine terms for the unsteady/turbulent 

concept 

20243 SI 
1478 S4 
25040 S8 
21177 S11 
10785 SI 4 
2784 SI 7 
2451 S20 
4584 S23 
70384 S24 
149936 S25 
734 S28 

540 232016 SI OR S4 OR S8 OR S11 OR Si 4 OR SI 7 OR S20 OR 

S23 OR S24 OR S25 OR S28 

?ss s40 or s31 or s34 or s35 or s39 
232016 S40 
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4812 S31 
29 S34 
4751 S35 
11472 S39 

S41 235453 S40 OR S31 OR S34 OR S35 OR S39 


?ss incompressib? 

Search the incompressible concept 

542 18388 INCOMPRESSIB? 

?ss compressib?()effect? 

543 15458 COMPRESSIB? 

544 454543 EFFECT? 

545 1437 COMPRESSIB?()EFFECT? 

?ss incompressib?()effect? 

546 18388 INCOMPRESSIB? 

547 454543 EFFECT? 

548 3 INCOMPRESSIB?()EFFECT? 

?ss incompressib?()flow? 

549 18388 INCOMPRESSIB? 

550 246544 FLOW? 

551 9317 INCOMPRESSIB?()FLOW? 

?ss high()reynold?()number? 

552 288546 HIGH 

553 27196 REYNOLD? 

554 138099 NUMBER? 

555 3039 HIGH()REYNOLD?()NUMBER? 

?ss incompressib?()boundar?()layer? 

556 18388 INCOMPRESSIB? 

557 135404 BOUNDAR? 

558 121533 LAYER? 

559 748 INCOMPRESSIB?()BOUNDAR?()LAYER? 

?ss incompressib?()fluid? 

560 18388 INCOMPRESSIB? 

561 95669 FLUID? 

562 5832 INCOMPRESSIB?()FLUID? 

?ss s42 or s45 or s48 or s51 or s55 or s59 or s62 

Combine synonyms for the incompressible 

concept 

18388 S42 
1437 S45 
3 S48 
9317 S51 
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3039 S55 
748 S59 
5832 S62 

563 22118 S42 OR S45 OR S48 OR S51 OR S55 OR S59 OR S62 
?ss bluff()bod? 

Search the bluff bodies concept 

564 1148 BLUFF 

565 93453 BOD? 

566 1039 BLUFF()BOD? 

?e nonli 

Expanding on the term nonlifting in the basic 

index 

Ref Items Index-term 

El 1 NONLEVEUNG 

E2 1 NONLEVITATED 

E3 1 *NONLI 
E4 1 NONLIDAR 

E5 1 NON LI EAR 

E6 1 NONLIENAR 

E7 1 NONLIFE 

E8 2 NONLIFT 

E9 244 NONLIFTING 

E10 8 NONLIFTING VEHICLE 

Ell 3 NONLIGHT 

E12 1 NONLIGHTLY 

?ss nonlift? 

567 246 NONLIFT? 

?ss non()lift? 

568 36901 NON 

569 19086 LIFT? 

570 59 NON()LIFT? 

?ss bluff()bod?()flow? 

571 1148 BLUFF 

572 93453 BOD? 

573 246544 FLOW? 

574 36 BLUFF()BOD?()FLOW? 

?ss ground?()effect? 

575 70051 GROUND? 

576 454543 EFFECT? 

577 3156 GROUND?()EFFECT? 


?ss s66 or s67 or s70 or s74 or s77 
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Combine synonyms for the bluff body/ground 
effect concept 


1039 S66 
246 S67 
59 S70 
36 S74 
3156 S77 

578 4486 S66 OR S67 OR S70 OR S74 OR S77 
?ss s41 and s63 and s78 

Combine the three concepts 

235453 S41 
22118 S63 
4486 S78 

579 165 S41 AND S63 AND S78 
?ss ud=9001 :9212b2 

Set the update for 1990-1992 

S80 205080 UD=9001 :9212B2 
?ss s79 and s80 

Limit by the update 

165 S79 
205080 S80 

581 26 S79 AND S80 

Due to small search set, decided to reexamine 
the relation of the concepts to each other. 
Current use of ANDing might be inappropriate 
due to the meaning of the terms and their 
relation to one another in the indexing 
language hierarchy. 

?ss finite analy()analy?()method? 

Unsure of the meaning of these terms and 
where they fit in the structure of the query 
so they were excluded (s85 and s89) from 
final sets 

582 76274 FINITE 

583 546777 ANALY? 

584 350376 METHOD? 

585 32 FINITE()ANALY?()METHOD? 

?ss finite()differen?()theor? 

586 76274 FINITE 

587 230475 DIFFEREN? 

588 304131 THEOR? 

589 15945 FINITE()DIFFEREN?()THEOR? 

?ss s41 or s63 

235453 S41 
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22118 S63 

S90 245019 S41 OR S63 


?ss s90 and s78 

Combine the unsteady concept and the bluff 
body concept 

245019 S90 
4486 S78 

591 1665 S90ANDS78 

?ss s90 or (s85 or s89 or (finite()element?) 

Include finite element in the unsteady 

concept 

245019 S90 
32 S85 
15945 S89 

592 76274 FINITE 

593 117011 ELEMENT? 

594 31200 FINITE(W)ELEMENT? 

595 278359 S90 OR S85 OR S89 OR (FINITE()ELEMENT?) 

?ss s95 and s78 

Combine the unsteady concept and bluff body 

concept 

278359 S95 
4486 S78 

596 1697 S95ANDS78 


?ss s80 and s96 

205080 S80 
1697 S96 

S97 241 S80 AND S96 


Limit by the update 


?ss s85 or s89 or s94 

Combine finite difference, finite element and 
finite analysis 

32 S85 
15945 S89 
31200 S94 

S98 45673 S85 OR S89 OR S94 


?ss s90 and s98 and s78 

Combine the unsteady concept, the finite 
difference/element/analysis concept, and the 
bluff body concept 


245019 S90 
45673 S98 
4486 S78 
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S99 95 S90 AND S98 AND S78 


?ss s99 and s80 

95 S99 
205080 S80 

S100 15 S99ANDS80 


Limit by the update 


?ss s97/de* 

SI 01 190 S97/DE* 


Sift for major descriptors 


?ss s97/de 

SI 02 234 S97/DE 


Sift for major or minor descriptors 


?ss s97 not si 01 

The NOTed sets are checking for duplicates. 
Used set 97 as the final set. 

241 S97 
190 S101 

SI 03 51 S97 NOT Si 01 


?ss si 01 not si 02 
190 S101 
234 SI 02 

S104 0 Si 01 NOT Si 02 

?ss si 02 not si 01 

234 SI 02 
190 S101 

SI 05 44 Si 02 NOT Si 01 

?ss s97 not (si 01 or si 02) 

241 S97 
190 S101 
234 Si 02 


SI 06 7 S97 NOT (Si 01 OR Si 02) 

?logoff 
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QUESTION 1 


Set Items Description 


?ss aerodynamic?()stall?/de 

51 51164 AERODYNAMIC7/DE 

52 2527 STALL7/DE 

53 1907 AERODYNAMIC?()STALL?/DE 


?ss stall?/de 

S4 2527 STALL7/DE 


?ss s3 or s4 

1907 S3 
2527 S4 

55 2527 S3 OR S4 

?ss computer()program?de 

56 202425 COMPUTER? 

57 0 PROGRAM 7DE 

58 0 COMPUTER?()PROGRAM?DE 

?ss computer?/()program?/de 

59 177936 COMPUTER7/DE 

510 125934 PROGRAM7/DE 

511 66375 COMPUTER?()PROGRAM?/DE 

?ss math()?()model?/de 
SI 2 154874 MATH7/DE 

51 3 177623 MODEL7/DE 

514 85604 MATH?()MODEL?/DE 

?ss sll or si 4 

66375 S11 
85604 SI 4 

SI 5 144914 S11 OR Si 4 

?ss turbulen?/de 

SI 6 62291 TURBULEN7/DE 

?ss boundar?()layer?/de 

SI 7 102138 BOUNDAR7/DE 
SI 8 66592 LAYER7/DE 

51 9 46540 BOUNDAR?()LAYER?/DE 

?ss unstead?/de 

520 13212 UNSTEAD7/DE 

?ss si 6 or si 9 or s20 
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62291 SI 6 
46540 SI 9 
13212 S20 

521 101505 SI 6 OR SI 9 OR S20 

?ss wind?/de 

522 83479 WIND7/DE 

?ss turbine?/de 

523 26313 TURBINE7/DE 

?ss rotor?/de 

524 14745 ROTOR7/DE 

?ss s22 or s23 or s24 
83479 S22 
26313 S23 
14745 S24 

525 117748 S22 OR S23 OR S24 

?c 5 and 15 and 21 and 25 
2527 5 
144914 15 
101505 21 
117748 25 

526 24 5 AND 15 AND 21 AND 25 

?ss ud«9001 :9212b2 

S27 205080 UD=9001 :9212B2 

?c 26 and 27 
24 26 
205080 27 

528 3 26 AND 27 

?ss s21 or flow? 

101505 S21 

529 251436 FLOW? 

530 282542 S21 OR FLOW? 

?ss s21 or flow?/de 
101505 S21 

531 205276 FLOW?/DE 

532 239863 S21 OR FLOW?/DE 

?c 5 and 15 and 32 and 25 
2527 5 
144914 15 
239863 32 
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117748 25 

533 52 5 AND 15 AND 32 AND 25 

?c 27 and 33 

205080 27 
52 33 

534 8 27 AND 33 

?ss s25 or wind?/ti 
117748 S25 

535 32419 WIND7/TI 

536 119527 S25 OR WIND7/TI 


?ss s32 or (turbulen?/ti or unstead?/ti) 

239863 S32 

537 33702 TURBULEN7/TI 

538 8260 UNSTEAD7/TI 

539 241305 S32 OR TURBULEN7/TI OR UNSTEAD7/TI 

?c 5 and 15 and 39 and 36 
2527 5 
144914 15 
241305 39 
119527 36 

540 54 5 AND 1 5 AND 39 AND 36 

?ss s3 or stall?()delay? 

1907 S3 

541 4156 STALL? 

542 18433 DELAY? 

543 26 STALL?(W)DELAY? 

544 1913 S3 OR STALL?()DELAY? 

?c 44 and 15 and 39 and 36 
1913 44 
144914 15 
241305 39 
119527 36 

545 42 44 AND 15 AND 39 AND 36 

?c 27 and 45 

205080 27 
42 45 

546 6 27 AND 45 

?ss aerodynamic?/de* or stall?/de 

547 31838 AERODYNAMIC7/DE* 

548 2527 STALL7/DE 

549 32857 AERODYNAMIC7/DE* OR STALL7/DE 
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?c 49 or 15 or 39 or 36 
32857 49 
144914 15 
241305 39 
119527 36 

550 444822 49 OR 15 OR 39 OR 36 

?ss s5 or stall?()delay? 

2527 S5 

551 4156 STALL? 

552 1 8433 DELAY? 

553 26 STALL?( W)D ELA Y? 

554 2531 S5 OR STALL?()DELAY? 

?c 54 and 15 and 39 and 36 
2531 54 
144914 15 
241305 39 
119527 36 

555 54 54 AND 15 AND 39 AND 36 

?c 27 and 55 

205080 27 
54 55 

556 8 27 AND 55 
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QUESTION 2 


Set Items Description 


?ss ud-9001 :9212b2 

SI 205080 UD=9001 :9212B2 

?ss modern()control()theor? 

52 9259 MODERN 

53 206837 CONTROL 

54 306568 THEOR? 

55 239 MODERN()CONTROL()THEOR? 

?ss h()2 

56 40803 H 

57 197069 2 

58 449 H()2 

?ss h()lnflnit? 

59 40803 H 

510 23500 INFINIT? 

511 653 H()INFINIT? 

?ss s5 or s8 or si 1 
239 S5 
449 S8 
653 S11 

512 1325 S5 OR S8 OR Si 1 

?ss robustness 

51 3 5387 ROBUSTNESS 

?ss robust?(3w)control 
SI 4 8098 ROBUST? 

SI 5 206837 CONTROL 

SI 6 1112 ROBUST?(3W)CONTROL 

?ss flight()control()system? 

SI 7 107537 FLIGHT 
SI 8 206837 CONTROL 
SI 9 513422 SYSTEM? 

520 2912 FLIGHT()CONTROL()SYSTEM? 

?ss multivariable control 

521 169 MULTIVARIABLE CONTROL 

?ss (control()synthesis)/ti 

522 61577 CONTROL/TI 
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523 10881 SYNTHESIS/TI 

524 212 (CONTROL()SYNTHESIS)/TI 

?ss (feedback control)/de* 

525 8826 (FEEDBACK CONTROL)/DE* 


?ss feedback control 

526 15751 FEEDBACK CONTROL 

?ss multivariable()feedback()control()system 

527 3069 MULTIVARIABLE 

528 26930 FEEDBACK 

529 206837 CONTROL 

530 340935 SYSTEM 

531 12 MULTI VARIABLE()FEEDBACK()CONTROL()SYSTEM 


?ss active()modal()suppression()system 

532 34732 ACTIVE 

533 9402 MODAL 

534 6541 SUPPRESSION 

535 340935 SYSTEM 

536 1 ACTIVE()MODAL()SUPPRESSION()SYSTEM 

?ss s20 or si 3 or s21 or si 6 or s24 
2912 S20 
5387 SI 3 
169 S21 
1112 S16 
212 S24 

537 8648 S20 OR Si 3 OR S21 OR Si 6 OR S24 

?ss s37 or s31 or s36 
8648 S37 
12 S31 
1 S36 

538 8657 S37 OR S31 OR S36 

?ss transport aircraft 

539 4754 TRANSPORT AIRCRAFT 


?e flexible aircraft 
Ref Items Index-term 
El 2 FLEXIBITY 

E2 14693 FLEXIBLE 
E3 0 'FLEXIBLE AIRCRAFT 
E4 3318 FLEXIBLE BODIES 
E5 98 FLEXIBLE BODY 

E6 1777 FLEXIBLE SPACECRAFT 

E7 45 FLEXIBLE WING 
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E8 297 FLEXIBLE WINGS 

E9 27 FLEXIBLEN 

E10 4 FLEXIBLER 

Ell 20 FLEXIBLES 

E12 2 FLEXIBLILITY 

?ss (flexible()bod? or wing?)/de 

540 5622 FLEXIBLE/DE 

541 61501 BOD7/DE 

542 3416 FLEXIBLE/DE(W)BOD?/DE 

543 25162 WING7/DE 

544 28500 (FLEXIBLEOBOD? OR WING?)/DE 

?ss (flexible()(bod? or wing?))/de 

545 5622 FLEXIBLE/DE 

546 61501 BOD7/DE 

547 25162 WING7/DE 

548 3754 (FLEXIBLE()(BOD? OR WING?))/DE 

?SS x()29 

549 95226 X 

550 6525 29 

551 296 X()29 

?ss lynx()helicopter? 

552 146 LYNX 

553 16531 HELICOPTER? 

554 67 LYNX()HELICOPTER? 

?ss maneuverable()aircraft()design? 

555 692 MANEUVERABLE 

556 121850 AIRCRAFT 

557 263297 DESIGN? 

558 2 MANEUVERABLE()AIRCRAFT()DESIGN? 

?ss s39 or s48 or s51 or s54 or s58 
4754 S39 
3754 S48 
296 S51 
67 S54 
2 S58 

559 8853 S39 OR S48 OR S51 OR S54 OR S58 

?c 12 and 38 and 59 
1325 12 
8657 38 
8853 59 

560 31 12 AND 38 AND 59 
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?c 60 and 1 

31 60 
205080 1 

561 25 60 AND 1 

?ss si 2 or (control()theory)/de* 

1325 SI 2 

562 105897 CONTROL/DE* 

563 68393 THEORY/DE* 

564 7289 CONTROL/DE*(W)THEORY/DE* 

565 8339 Si 2 OR (CONTROL()THEORY)/DE* 

?c 65 and 38 and 59 
8339 65 
8657 38 
8853 59 

566 65 65 AND 38 AND 59 

?c 1 and 66 

205080 1 
65 66 

567 42 1 AND 66 

?ss s67 not s61 
42 S67 
25 S61 

568 17 S67NOTS61 
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QUESTION 3 


Set Items Description 


?ss grid?()generat?/de* 

51 4795 GRID7/DE* 

52 22734 GENERAT7/DE* 

53 1243 GRID?()GENERAT?/DE* 

?ss finit?()different?()theor?/de 

54 47451 FINIT7/DE 

55 67381 DIFFEREN7/DE 

56 154251 THEOR7/DE 

57 16218 FINIT?()DIFFEREN?()THEOR?/DE 

?ss finit?()element?()method?/de 

58 47451 FINIT7/DE 

59 54517 ELEMENT7/DE 

510 105570 METHOD7/DE 

511 29106 FINIT?()ELEMENT?()METHOD?/DE 

?ss s7 or si 1 

16218 S7 
29106 S11 

51 2 43955 S7 0RS11 

?ss s3 and si 2 
1243 S3 
43955 SI 2 

51 3 360 S3 AND SI 2 

?ss ud=9001:9212b2 

S14 205080 UD=9001:9212B2 

?ss si 4 and si 3 
205080 SI 4 
360 S13 

515 236 S14 AND S13 

?ss s7/de* 

51 6 6363 S7/DE* 

?ss sll/de* 

S17 16425 SI 1/DE* 


?ss si 6 or si 7 
6363 SI 6 
16425 SI 7 

S18 22413 S16 OR S17 
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?ss s3 and si 6 

6363 SI 6 
1243 S3 

S20 52 S3 AND SI 6 


?ss s14 and s20 

205080 S14 
52 S20 

S21 37 S20 AND S14 

?ss s21 not computat?()fluid?()dynamic? 

37 S21 

522 82336 COMPUTAT? 

523 97938 FLUID? 

524 171698 DYNAMIC? 

525 26184 COMPUTAT?(W)FLUID?(W)DYNAMIC? 

526 23 S21 NOT COMPUTAT?()FLUID?{)DYNAMIC? 
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QUESTION 4 


?ss flow? or wake? or turbulen? or stabilit? 

52 248235 FLOW? 

53 12907 WAKE? 

54 70918 TURBULEN? 

55 136185 STABILI? 

S6 383912 FLOW? OR WAKE? OR TURBULEN? OR STABILI? 

?ss s6/ti 

57 146622 S6/TI 

?ss bluff()bod?/ti 

58 453 BLUFF/TI 

59 24657 BOD?/TI 

S10 387 BLUFF()BOD?/TI 

?ss s7 and slO 

146622 S7 
387 S10 

S11 286 S7ANDS10 

?ss ud=9001 :9212b2 

S13 205080 UD=9001 :9212B2 

?ss sll and si 3 
286 S11 
205080 SI 3 

S14 38 Sll AND Si 3 

?ss computat? or computational () fluid () 
dynamic? or reynold's () number 

51 5 80575 COMPUTAT? 

516 52140 COMPUTATIONAL 

51 7 81766 FLUID 

518 169087 DYNAMIC? 

SI 9 25391 COMPUTATIONAL(W)FLUID(W)DYNAMIC? 

520 0 REYNOLD'S 

521 127151 NUMBER 

522 0 REYNOLD'S(W)NUMBER 

523 80575 COMPUTAT? OR COMPUTATIONAL()FLUID()DYNAMIC? OR 

REYNOLD‘S()NUMBER 


? e reynold 

El 1 REYNODS 

E2 4 REYNOL 

E3 174 'REYNOLD 

E4 27314 REYNOLDS 

E5 786 REYNOLDS EQUATION 

E6 15690 REYNOLDS NUMBER 

E7 75 REYNOLDS NUMBER EFFECT 
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E8 2540 REYNOLDS STRESS 

E9 2 REYNOLDSA 

E10 1 REYNOLDSD 

Ell 2 REYNOLDSOVA 

E12 1 REYNOLDSOVO 

?ss e4,e7 

524 27314 REYNOLDS 

525 75 REYNOLDS NUMBER EFFECT 

526 27314 E4.E7 

?ss si 5 or (si 5 or si 9 or s26)/ti 

527 12189 S15/TI 

528 529 S19/TI 

529 3926 S26/TI 

530 16054 (S15 OR Si 9 OR S26)/TI 

?ss s30 and si 3 

16054 S30 
205080 SI 3 

531 2449 S30 AND SI 3 

?ss s7 and s31 

146622 S7 
2449 S31 

532 824 S7ANDS31 

?ss s32 and si 3 
824 S32 
205080 SI 3 

533 824 S32 AND Si 3 



Appendix C. WAIS Search Strategies 


72 



WAIS Terse Strategies 


QUESTION 1 

unsteady aerodynamics stall delay model air flows propeller 

QUESTION 2 

flight control system modem control theory flexible aircraft 

QUESTION 3 

adaptive gridding grid generation mesh redistribution truncation error structured flow mechanics 

QUESTION 4 

computational fluid dynamics bluff body 
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WAIS Verbose Strategies 
QUESTION 1 


unsteady state flow aerodynamics pressure turbulence flow wake model effects boundary layers 

mathematical models computer models digital simulation high solidity propellers compressed rotor 
stator blades wind power generator 


QUESTION 2 


modem control theory flight control design flight control active control h-infinity h-2 control system 
esign flexible aircraft body model dynamic model transport aircraft control configured vehicle" 


QUESTION 3 


gnd adaptive gnddmg mesh redistribution structural gridding grid generation computational fluid 
dynamics adaptive control flow distribution finite difference theory truncation error numeric analysis 
numerical analysis mathematical models math model mathematical model coordinate coordinates 

r:re c rr: 8 e th p s blcm so,v,ng approximate approximati ° n «■* 


unsteady state computational fluid dynamics turbulence flow turbulent flow unsteady flow unsteady 

mTbulpnrP l C ff tU f rbU en f t f wake turbu,ent wakes turbulent model turbulence pressure unsteady pressure 
turbulence effect nonstationaiy pressure nonstationaiy turbulence turbulence boundary layer 

incompressible compressible effect incompressible flow high reynolds number incompressible 
oundary layer incompressible fluid bluff bodies nonlifting non-lifting bluff body flow ground effect 
finite analysis method finite difference theoiy 
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